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This research investigated the relationship between online students' behaviors in a 
learning management system and their learning performance, as measured by their 
final grades. Four online information technology classes in a major U.S. library and 
information science program were studied. The course sites' log data, as well as the stu¬ 
dents' final grades were analyzed at both the class level and the general level using the 
aggregated data from all four classes. The results show that the number of days students 
delayed in accessing weekly lecture materials had a significantly negative correlation 
with the final grade, while an increased number of discussion board postings had a 
positive correlation with the final grade. This correlation was significant on borderline, 
at p = 0.05 level. In one class an increase in the frequency of accessing the syllabus 
was significantly correlated with the final grade. The implications of the results are dis¬ 
cussed. Based on the findings, suggestions on strengthening e-learning are proposed. 
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Introduction 

Background 

O nline education, supported by vari¬ 
ous forms of technologically rich 
platforms, mainly learning management 
systems that integrates student manage¬ 
ment, learning material delivery, learning 
assessment, etc., has increased rapidly in 
recent years. According to the most recent 
report on online education in the United 
States by Babson Research Group (Al¬ 
len, Seaman, Poulin, & Straut, 2016), 
which uses the U.S. National Center for 
Education Statistics data, by fall 2014, in 
the U.S. there was a 3.9% increase in the 
number of distance education stu¬ 
dents, up from the 3.7% rate recorded the 
previous year. More than one in four stu¬ 
dents (28% of a total of 5,828,826 on¬ 
line students) was taking at least one dis¬ 


tance education course. Among the 
5.8 million online students, 2.85 million 
were taking all of their courses at a dis¬ 
tance and 2.97 million taking some, but not 
all, distance courses (Allen et al., 2016). 

In the field of library and information 
science, Islam, Kunifuji, Hayama, and Mi- 
ura (2011) found that 85 (about 23% of a 
total of370 worldwide) LIS programs were 
using e-learning systems for delivering ed¬ 
ucation, and Blackboard/Blackboard Vista 
(25.89% was the most frequently used 
learning management system). The major¬ 
ity (i.e., 73 of 85 LIS programs; 86%) were 
delivering their course contents asynchro¬ 
nously (Islam et al., 2011). More recently, 
according to the American Library Asso¬ 
ciation (ALA) (2016), among the 59 ALA 
accredited schools, over the half (30) now 
offer 100% online programs. Only eight 
schools do not have any online offering. 
The remaining schools offer online educa- 


J. of Education for Library and Information Science, Vol. 57, No. 4—(Fall) October 2016 
ISSN: 0748-5786 © 2016 Association for Library and Information Science Education 
doi: 10.12783/issn.2328-2967/57/4/1 


255 


256 


JOURNAL OF EDUCATION FOR LIBRARY AND INFORMATION SCIENCE 


tion for select courses (ALA, 2016). 

Online education in general provides 
several advantages when compared with 
traditional face-to-face classroom set¬ 
tings. For example, online students have 
the flexibility in terms of time and loca¬ 
tion to access the course material, and to 
participate in class discussions, as long 
as they have access to the needed tech¬ 
nology (e.g., Barcelona, 2009; Fincham, 
2013; Flampel, 2014). This flexibility is 
particularly important for those who want 
to continue their education while working 
part- or full-time. Studies have demon¬ 
strated that e-leaming is welcomed by stu¬ 
dents when supported by a learning man¬ 
agement system (e.g., Dang & Robertson, 
2010; Juhary, 2014; Lonn & Teasley, 
2009; Martin-Bias & Serrano-Femandez, 
2009). Nguyen (2015) found that online 
learning is generally at least as effective as 
the traditional format. 

An online course typically delivers 
course material digitally to the class using 
a learning management system. Based on 
the media through which online education 
is delivered, online education/leaming 
can be synchronous or asynchronous. The 
former is commonly supported by media 
such as videoconference and live chat, 
and the latter is commonly facilitated by 
media such as email and discussion board 
(Flrastinski, 2008a). In either setting, the 
instructor’s role has changed from lecturer 
to facilitator in students’ learning. 

The current study focuses on asynchro¬ 
nous e-leaming. The study examines the 
relationships between students’ online be¬ 
haviors on a learning management system 
and their learning performance, to identify 
the behavior patterns affecting learning 
performance and to suggest ways to im¬ 
prove student learning in the online envi¬ 
ronment. 

Research Questions 

This study focused on online learn¬ 
ing using a learning management system: 
the Blackboard system. Robert Godwin- 


Jones (2012) described a typical Black¬ 
board model, which consisted of an online 
gradebook, posted syllabus, assignment 
tracking, linked and/or uploaded course 
materials, and threaded discussion boards 
for interactivity and collaboration. This is 
a typical asynchronous online learning en¬ 
vironment. This study focused on examin¬ 
ing students’ online learning behaviors in 
accessing the course syllabus and weekly 
posted learning materials, and through 
their posting to the learning management 
system’s discussion board. The interaction 
activities investigated in this study, (i.e., 
accessing course syllabus, weekly posted 
learning materials by the instructor, and 
communicating with the class via the sys¬ 
tem’s discussion board), are fundamen¬ 
tal activities on a learning management 
system that every student has to perform 
when taking an online course. How did 
these fundamental activities impact stu¬ 
dents’ learning was examined here. 

The research questions for the study, 
are therefore: 

Ql. Does student access to the course syl¬ 
labus relate to learning performance? 

Q2. Does student access to weekly learn¬ 
ing materials posted by the instructor 
relate to learning performance? 

Q3. Does the use of the discussion board 
on the course site relate to learning 
performance? 

Literature Review 

Research on students’ general use 
(without distinguishing synchronous and 
asynchronous learning) of IT in learning 
has produced mixed results. While Dang 
& Robertson (2010), Juhary (2014), Lonn 
& Teasley (2009), and Martin-Bias & Ser¬ 
rano-Femandez (2009) showed that stu¬ 
dents preferred e-leaming using learning 
management systems, Margaryan, Little¬ 
john, & Vojt (2011) found that students 
appeared to select traditional pedagogies, 
with only minor uses of IT tools for de¬ 
livering content. The latter study therefore 
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suggests that it would be misleading to 
claim that students were shifting the pat¬ 
terns of learning and were in favor of tech¬ 
nology use. 

Several studies have investigated the 
use of learning management systems and 
students’ learning. Spivey and McMillan 
(2013) studied the relationship between 
student effort and performance among stu¬ 
dents in a finance course, using the Black¬ 
board system’s logged data. Student ef¬ 
fort was measured by the number of times 
students accessed learning materials that 
were placed on the Blackboard system. 
The study examined if more frequent ac¬ 
cess of the resources would affect the stu¬ 
dents’ grade performance. The results of 
the study indicate that performance was 
positively influenced by the frequency of 
accessing. A more evenly spaced study 
schedule was also found to have a positive 
effect on performance, while cramming 
(i.e., intense studying in a short period of 
time immediately preceding an exam) did 
not. The mode of course delivery for this 
study, however, appears to be a blended 
learning context, where course materials 
were delivered via the Blackboard system, 
but tests were conducted within a class¬ 
room. This context is quite different from 
a purely online learning setting, where 
students are located remotely at locations 
far away from each other and from the in¬ 
structor. 

Liang, Jia, Wu, Miao, and Wang (2014) 
studied the factors that might influence 
learners’ participation and performance in 
a MOOC environment. Students’ learning 
records captured in the learning manage¬ 
ment system and their feedback collected 
from a survey were explored. Regres¬ 
sion analysis was conducted to examine 
the correlation among perceived learning 
experience, learning activities and learn¬ 
ing outcomes. Their findings suggest that 
learners’ perceived usefulness rather than 
perceived ease of use of the MOOC posi¬ 
tively influenced learners’ use of the sys¬ 
tem, and consequentially, their learning 
outcome (Liang et al., 2014). 


Hung, Chou, Chen, and Own (2010) de¬ 
veloped an online learning readiness scale 
to measure students’ readiness for online 
learning. The scale included five dimen¬ 
sions: self-directed learning, motivation 
for learning, computer/Intemet self-ef¬ 
ficacy, learner control, and online com¬ 
munication self-efficacy. Data from 1051 
college students in five online courses in 
Taiwan revealed that students’ levels of 
readiness were high in computer/Intemet 
self-efficacy, motivation for learning, and 
online communication self-efficacy and 
were low in learner control and self-di¬ 
rected learning. This study found that stu¬ 
dents in a higher grade (junior and senior) 
exhibited significantly greater readiness in 
the dimensions of self-directed learning, 
online communication self-efficacy, mo¬ 
tivation for learning, and learner control 
than did students in lower grades (fresh¬ 
man and sophomore) (Huang et al., 2010). 

In addition to the effect of student effort 
of accessing the learning materials posted 
on the course site, a few studies investi¬ 
gated the use of discussion boards on the 
Blackboard system and reported varying 
results. Alghamdi (2013) reported that the 
frequency of posting to Blackboard dis¬ 
cussion board was positively related to im¬ 
provements seen in the test scores among 
those in the control group but not among 
those in the experimental group. Badawy 
& Hugue (2010) reported a positive attitude 
from students regarding the use of discus¬ 
sion boards in their online courses. How¬ 
ever, most studies which examined the use 
of discussion boards in the online learning 
environment focused on students’ prefer¬ 
ence or satisfaction, rather than on the po¬ 
tential impact of students’ discussion board 
behavior on their learning performance. 

When comparing synchronous and 
asynchronous online learning, Coogle 
and Floyd (2015) found that education 
graduate students in their study perceived 
benefits from both synchronous and asyn¬ 
chronous learning environments. Hras- 
tinski (2008b) found different effects of 
synchronous and asynchronous online dis- 
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cussions, by examining two online classes 
that participated in two asynchronous and 
two synchronous online discussions. Syn¬ 
chronous communication induced person¬ 
al participation, which could be regarded 
as a complement to cognitive participa¬ 
tion. However, cognitive participation is 
a more reflective type of participation 
supported by asynchronous communi¬ 
cation. In synchronous discussions, the 
e-leamers felt that they worked together 
and were not restricted to only discuss 
course content. This was likely to induce 
arousal and motivation and increased con¬ 
vergence on meaning, especially in small 
groups (Coogle & Floyed, 2015). Clouse 
& Evans (2003) investigated the effects 
of asynchronous and synchronous lec¬ 
tures and discussions with graduate busi¬ 
ness program classes both on campus and 
off campus. Their results indicated that in 
three scenarios, synchronous face-to-face 
or videoconference lecture/synchronous 
face-to-face or videoconference discus¬ 
sion, synchronous face-to-face or video- 
conference lecture/asynchronous threaded 
discussion, and asynchronous online re¬ 
corded PowerPoint lecture/asynchronous 
threaded discussion, the student perfor¬ 
mance on a test was similar. Only when an 
asynchronous online recorded PowerPoint 
lecture was combined with a synchronous 
online chat discussion, the student perfor¬ 
mance was significantly lower than others. 

The previous research on the differ¬ 
ences between synchronous and asyn¬ 
chronous e-learning indicate that each 
has its own advantages and limitations. 
While synchronous e-leaming enhances 
students’ feeling of participation, enables 
them to get immediate feedback, and helps 
communications, it reduces the flexibil¬ 
ity in terms of time the online students 
would enjoy. Also, as a learning tool, 
synchronous chatting includes an unorga¬ 
nized manner that everyone could speak 
at once at a discussion session. Asynchro¬ 
nous learning, on the other hand, can en¬ 
able more students in discussion and can 
provide students with time to reflect on 


content and responses from others prior 
to posting their own response (Clouse & 
Evans, 2003). Asynchronous e-leaming 
also fully supports the flexibility the on¬ 
line education offers. 

Given the mixed results regarding the 
general use of technology in learning, and 
the lack of detailed analysis on the effect 
of behaviors on learning performance in 
online learning settings, the current re¬ 
search sought to use students’ access data 
on the Blackboard system that was used 
mainly as an asynchronous e-leaming 
platform, to explore whether students’ 
interaction behaviors are associated with 
learning performance. Different from the 
study conducted by Spivey and McMillan 
(2013), the current research focused on 
online students living in geographically 
dispersed locations, where the course site 
served as the primary place they would 
meet asynchronously. The study used ob¬ 
servable behavioral measures instead of 
proxy measures such as satisfaction or 
preference that are discussed in Naveh, 
Tubin and Pliskin (2010). 

Methodology 

This research employed the learning 
analytics approach introduced by Jones 
(2012). The students’ course access data 
was analyzed to reveal their interaction 
behaviors on the learning management 
system course sites. The data was pre- 
processed, and then analyzed on identi¬ 
fied important measures, to answer the re¬ 
search questions established for this study. 

Course Settings in the Study 

Two information technology (IT) 
courses in a master of library and informa¬ 
tion science program in the U.S. were in¬ 
volved in this study. Of these two courses, 
one was a mandatory introductory course. 
The other course was an advanced, elec¬ 
tive course, taken by the students mainly 
based on their interests. These two courses 
are regularly offered every semester. The 
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classes in fall 2014 (F2014) (September- 
early December 2014) and winter 2015 
(W2015) (early January-end of April, 
2015) semesters were examined, leading 
to a total of four separate classes (each of 
the two courses had one class in each of 
the semesters). 

The two IT courses involved in the cur¬ 
rent study were organized following the 
model described by Robert Godwin-Jones 
(2012). The major course components on 
the course site included course announce¬ 
ment, course syllabus, contents, discussion 
board, and assignments. Each of the com¬ 
ponents could be accessed from the navi¬ 
gation pane on the course site. The course 
announcement area was set as the default 
display when the course was accessed. 

Course syllabi were posted at the be¬ 
ginning of the semester a few days before 
the classes officially started. Opening the 
syllabus link would lead directly to the 
display of the PDF version of the course 
syllabus on the screen. 

Weekly learning materials such as lec¬ 
ture notes, tutorials, videos, readings, etc. 
were located in a “Contents” folder, and 
these were further organized into folders 
by weeks. Class materials were posted by 
the instructor week by week, on a specific 
day (such as Monday) of the week as the 
semester progressed. 

For all classes, assignments were posted 
in an “Assignments” folder with a link to it 
from the course home page. All assignment 
description files, except the final exam for 
the introduction class which was hosted on 
the course site and could be done only on 
the course site, could be downloaded by 
students and submitted to the course site 
after an assignment was completed, before 
or on the due date. Similar to the weekly 
class learning materials, assignments were 
also posted progressively along the class 
schedule, usually a few weeks before the 
due date. 

Discussion forums on the discussion 
board were created by the instructor, with 
a few types: the regular ones included 
weekly discussions for each week’s con¬ 


tent related topics and question and an¬ 
swers for each major assignment. Addi¬ 
tional forums were created in response to 
special class needs, such as calling for stu¬ 
dents’ willingness to have a synchronous 
online meeting, or forming small study 
groups, etc. When discussing on content 
related topics, students were required to 
post meaningful comments. Postings like 
“I agree,” “me, too,” or “absolutely,” etc., 
were not considered quality postings, and 
were explicitly prohibited in the course 
syllabus. Therefore, student postings did 
not have such short one or two words 
postings. For each week two students were 
assigned to facilitate discussion. Their re¬ 
sponsibilities included posting discussion 
topics, normally two or three, that are re¬ 
lated to the content and to follow up with 
the rest of the class’ responses, if needed. 
The instructor monitored the discussions, 
and would join discussions from time to 
time. Forums were created as the semester 
went by. For each of the four classes, par¬ 
ticipation in class discussion accounted for 
10% of the total grade for the course. 

The discussion board allows for stu- 
dent-to-student; student-to-instructor and 
instructor-to-students asynchronous com¬ 
munications. Previous studies on the use 
of discussion board mainly focused on 
students’ preference or satisfaction (Ser- 
vonsky, Daniels, & Davis, 2005). In this 
study, we wanted to find out if discussion 
board activity would directly relate to the 
students’ learning performance. 

While blended learning is adopted 
widely in higher education, which mixes 
elements of face-to-face and online edu¬ 
cational delivery (Agosto, Copeland, & 
Zach, 2013), the courses involved in the 
current research were completely online. 
Occasionally Adobe Connect, a synchro¬ 
nous teleconference tool would be used 
to have an online discussion session or a 
guest lecture, but such use was rare and 
was skipped in this research. The classes 
were chosen in the study for the conve¬ 
nience reason because the author was the 
instructor, who was a faculty member with 
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more than 10 years’ experience teaching 
online. The instructor was fully engaged 
in the classes, monitoring the course sites 
daily, often several times a day, to follow 
the progress of the classes. 

Students 

The four classes included a total of 88 
students who completed the course. The 
general statistics about the students are 
presented in Table 1. Since no students’ de¬ 
mographic data was available in the learn¬ 
ing management system, students’ gender 
was identified based on their names. 

The numbers in Table 1 do not include 
those who withdrew from the class dur¬ 
ing the semester, but they do include a 
few who failed the class at the end, i.e., 
it includes all students who completed the 
course, whether successful or not. 

The introduction course was a required 
course for all students. A student could be 
waived out of the course if the student had 
prior computing course work or experi¬ 
ence, and was familiar with the applica¬ 
tions and concepts covered in the course, 
as evidenced by passing a quick test. The 
advanced course was an elective course 
for those who wanted to enhance their IT 
skills. The majority of the students were 
female students, as can be seen in Table 

1. Only seven were male students. All stu¬ 
dents taking these classes were required to 
have access to high speed internet and ap¬ 
propriate computers, as a condition to be 
accepted into the online program. 


Table 1. General Statistics on the 
Student Samples. 


Semester 

Male 

Female 

Total 

F2014-Intro 

1 

22 

23 

W201 5-Intro 

2 

20 

22 

F2014-Advanced 

1 

21 

22 

W201 5-Advanced 

3 

18 

21 

Total 

7 

81 

88 


Data Set 

The data for this research was collected 
from three sources: 

1. The students’ final grade data was ac¬ 
cessed directly from each of the four 
class course sites. 

2. The discussion board data was also 
from each class site, but this data was 
included in a report for the instructor 
created by the Blackboard system to 
examine the number of postings on the 
discussion board. The report included 
the number of postings distributing 
month by month over the semester, fo¬ 
rum by forum; and the total number of 
postings by each student; 

3. The students’ course material access 
data was extracted from the Black¬ 
board system’s log files, provided by 
the Blackboard system’s campus sup¬ 
port team. The Blackboard system’s 
log data contained student IDs, the 
Blackboard content accessed, and the 
timestamps of access. 

Students ’ Interaction Behavior 
Measures 

Human-computer interaction behaviors 
may be measured from different dimen¬ 
sions, depending on the purpose of the 
research and the granularity of the avail¬ 
able data. After a careful examination of 
the data used for this study, the following 
measures were identified for the learning 
interaction behaviors investigated in this 
study: 

• For course syllabus access, the frequen¬ 
cy of accessing was used in the study, 
(i.e., the number of times a student 
accessed the course syllabus during 
the semester). Given the role of the 
courses’ syllabus in a class, it would be 
expected that access frequency is linked 
to the familiarity about the course 
organization and schedule, and thus is 
linked to learning activity management, 
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and ultimately learning performance. 
Increased access to the course’s syl¬ 
labus would lead to more organized 
learning which would ultimately help 
a student achieve a better final grade. 
However, it was also possible that the 
syllabus file was downloaded onto the 
student’s computer for access. In these 
instances when access to the syllabus 
occurred on a local computer, the figure 
could not be counted. 

• For accessing weekly posted learning 
materials, two measures were used: 

a. The number of weeks that the student 
failed to access posted weekly learn¬ 
ing materials. This measure is simi¬ 
lar to the number of classes a student 
would miss/skip in a traditional face- 
to-face classroom setting. Students 
were supposed to go over the posted 
learning materials each week as part 
of the learning process. This measure 
was intended to find out whether the 
students on a learning management 
system would also stick to the course 
schedule and follow course learning 
materials week by week. 

b. The timeliness of accessing weekly 
learning materials, or the number of 
days between the day when the learn¬ 
ing materials posted and the day the 
student actually accessed the posted 
materials, if that week was not com¬ 
pletely skipped by the student. For 
example, “0” means the student ac¬ 
cessed the materials the same day 
when the materials were posted by 
the instructor, and “35” would mean 
that the student accessed the materi¬ 
als 35 days after the materials were 
posted. 

One important requirement for online 
students is to be self-disciplined. This in¬ 
cludes following the class schedule and 
studying the learning materials along the 
timeline of the course. The class learning 
materials were distributed every week on 
the scheduled day specified in the course 


syllabus, similar to the class meeting in a 
face-to-face situation. The above two mea¬ 
sures were intended to determine whether 
students would access the learning mate¬ 
rials on the course site, and whether their 
access was on schedule. The idea was that 
the sooner the student accessed a week’s 
materials, the closer the student followed 
the schedule and that these behaviors 
would help improve student learning. 

• For using the discussion board, the 
total number of postings (all were 
qualified ones) by each student on 
the discussion board was used as the 
measure for participation or commu¬ 
nication in class. This measure reflects 
how involved a student was in his or 
her learning. 

Enabling students to communicate on¬ 
line on the course site is one of the basic 
requirements of a learning management 
system. While the use of a discussion 
board has been shown to have a posi¬ 
tive impact on student satisfaction with 
e-leaming (Badawy & Hugue, 2010), the 
current study was particularly interested in 
finding out whether the use could be di¬ 
rectly related to students’ learning perfor¬ 
mance. 

Amount of time spent on a task in an 
online context has been a frequently used 
measure in studying human behaviors. For 
example, the amount of time spent reading 
a document may be used as an important 
indicator of how much the reader under¬ 
stood the document. This measure, how¬ 
ever, is only partially reliable in settings 
where online behavior cannot be directly 
observed. In log data this is an unreliable 
measure because what a user was actually 
doing during that period of time is unclear. 
For example, a document could be opened 
for hours or even days but this does not 
necessarily mean the user actually read the 
accessed document for that long. There¬ 
fore, the amount of time on task was not 
used as a measure in this study. 

For accessing the weekly posted learn¬ 
ing materials, the frequency of access, 
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which had been used in Spivey and Mc¬ 
Millan (2013), was not used in this study 
because some materials were in com¬ 
pressed format, and the students needed to 
download such files to their local comput¬ 
er. Unlike the situation with accessing to 
the course syllabus, where there was just 
a possibility that the file could be down¬ 
loaded, in this case, it was certain that part 
of weekly learning materials had to be 
downloaded to the local computer. In such 
situations, the access would be to their lo¬ 
cal disk, which are unknown to the study. 

Learning Performance 

The student’s final grade was used as 
the measure of learning performance. The 
final grade received by each student was 
an accumulated score for his or her per¬ 
formance in the entire course. The score 
ranged from 0 to 100. 

Data Processing and Analysis 

Data collected from the different sourc¬ 
es noted above were first cleaned, by re¬ 
moving the data that was linked to the 
students who withdrew from the class. 
Pearson correlation analysis was the main 
statistical procedure to analyze the rela¬ 
tionships between different measures of 
learning interaction behaviors and the fi¬ 
nal grade. The data analysis was first per¬ 
formed on each class’s data, hoping to find 
the effects of the behavior measures in 
each class. The data from the four classes 
were then integrated into one aggregated 
data set. The analysis on this data set was 
undertaken at a general level rather than 
focusing on an individual class. 

Results 

The descriptive statistics on students’ 
final grades are presented first. The re¬ 
sults from the analysis for each of the four 
classes are presented the next, and this is 
followed by the results from the analysis 
on the aggregated data set. 


Table 2. Summary of Student Grades 
for the Involved Classes. 


Class-Semester 

Average 

Std. Dev. 

N 

lntro-F2014 

89.7 

9.4 

23 

lntro-W201 5 

89 

9.6 

22 

Advanced-F2014 

94.7 

3.1 

22 

Advanced-W201 5 

95 

3.8 

21 


Studen ts ’ Final Grades for Different 
Classes 

The data in Table 2 shows that for the 
introductory classes the students per¬ 
formed well overall, but that their grades 
varied dramatically, with an average dif¬ 
ference around 9.5 points. However, for 
the advanced classes, the students’ grades 
averaged about 95 points, with differences 
being the grades less than 4 points. This 
grade improvement may be explained by 
the fact that the students who took the 
advanced class had been trained in the 
introduction class, and thus had certain 
familiarity with the content of the ad¬ 
vanced class. More importantly, since the 
advanced course was an elective course, 
not a required one, the students were more 
interested and were more motivated in the 
advanced class, and thus did better than in 
the introductory class. 

Class Level Analyses 

To find out whether there would be sig¬ 
nificant correlations in a class, data analy¬ 
sis was first performed separately for each 
of the four classes. Different measures 
from each class were examined first. The 
descriptive statistics of the four classes are 
presented in Table 3. The numbers in the 
table are averages and standard deviations 
in parenthesis. 

As Table 3 shows, in terms of the num¬ 
ber of times the course syllabus was ac¬ 
cessed, the introductory classes were 
slightly more than the advanced classes. 
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Table 3. Descriptive Statistics on the Behavior Variables of the Four Classes. 


Class 

# of Syllabus Access 

# of Skipped Weeks 

# of Delayed Days 

# of Postings 

lntro-F2014 

14.2* (11.1)** 

2.8 (2.5) 

4.3 (2.5) 

56 (30) 

lntro-W201 5 

11.8 (8.4) 

1.3 (1.1) 

5.6 (4.2) 

34.2 (11.6) 

Advanced-F2014 

9.5 (4) 

2.6 (2.7) 

5.5 (3.7) 

43 (30.4) 

Advanced-W201 5 

10.2 (6.6) 

6.2 (2.5) 

5.3 (3.6) 

60.5 (60.3) 


Notes: ‘Averages; “Standard deviations 


Different classes have different dynam¬ 
ics. Student behaviors can be dramati¬ 
cally different from class to class, even 
for the same course. This is reflected in 
the number of weekly lectures skipped by 
students. While the students in the winter 
2015 Introductory class skipped only one 
and about one third weeks on average, the 
students in the winter 2015 advanced class 
skipped 6.2 weeks on average, as Table 3 
shows. Even comparing between the two 
classes of the same course, the winter 
2015 advanced class skipped more weeks 
than the fall 2014 class did. 

Different from the variations in the 
number of skipped weeks, the number 
of days delayed to access weekly posted 
learning materials seemed to be in the 
same range across the four classes. Aver¬ 
agely the students would delay about 4 or 
5 days to check the posted materials. 

There are also variations in the number 
of postings among the different classes. 
The winter 2015 introductory class post¬ 
ed only 34 times on average per student, 
the advanced class, also in the winter se¬ 
mester, posted over 60 times per student, 


nearly doubled the number of postings the 
introductory class did. 

The results of Pearson correlation anal¬ 
ysis are presented in Tables 4, for each 
class respectively. 

In Table 4, the numbers listed under 
each behavior measure are Pearson cor¬ 
relation coefficients with the final grade 
for each class. In each cell that contains 
numbers, the first number is correlation 
coefficient and the number following in 
the parenthesis is the significance p value. 
Among the four classes, only in the win¬ 
ter 2015 Introduction class the correlation 
between the number of times the syllabus 
accessed and the final grade is significant, 
at p = 0.049. The correlation is positive: 
the more access to the syllabus the higher 
the final grade. 

In other classes, there are no significant 
correlations between the two measures. 
Some low-grade students had high number 
of accesses and some high-grade students 
had very low number of accesses. 

Did skipped and delayed access to the 
weekly posted learning materials relate to a 
student’s final grade? Correlation analyses 


Table 4. Correlations between Behavior Measures and 
Final Grade for Four Classes. 


Class 


# of Syllabus 
Access 

# of Skipped 
Weeks 

# of Delayed 
Days 

# of Postings 

lntro-F2014 

Correlation: 

0.069 (0.377*) 

-0.152 (0.244) 

-0.229 (0147) 

0.326 (0.065) 

lntro-W201 5 

Correlation: 

0.361 (0.049) 

0.085 (0.354) 

-0.318 (0.074) 

0.283 (0.101) 

Advanced-F2014 

Correlation: 

-0.043 (0.424) 

-0.198 (0.188) 

-0.073 (0.374) 

-0.255 (0.126) 

Advanced-W201 5 

Correlation: 

0.069 (0.384) 

-0.211 (0.180) 

-0.325 (0.075) 

0.246 (0.141) 


Notes: ‘Significance (p), one-tailed 
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found no significant correlations between 
the two measures and the final grade. It did 
not seem to matter whether or not an on¬ 
line student’s skipping or delayed access 
to some weekly lectures would affect the 
student’s grade, in any of the four classes. 
However, the correlation coefficients are 
indeed negative (except the winter intro¬ 
ductory class, which is just less than 0.1), 
indicating a reverse relationship between 
the two measures and the final grade: the 
more skipped weeks or longer delays the 
lower the final grade. This relationship 
makes sense because without accessing (in 
a timely fashion) to learning materials the 
learning performance would not be good. 

No significant correlation was found 
either between the number of postings 
and the final grade, in any of the four 
classes. A student who posted minimum 
number of postings could still get a high 
grade, and a student who posted many 
times could still get a low grade. The cor¬ 
relations mostly are positive, meaning 
the more postings the higher the grade. 
In one class, however, the correlation is 
a negative one. This was probably due to 
the small data set. 

Analysis at the General Level on the 
Aggregated Data Set 

In this analysis, the data for the four 
classes was treated as one data set. By put¬ 
ting all data together, it was hoped that the 
data size would be large enough to make 
predictions. Pearson correlation analysis 
was conducted to examine the correlations 
between various measures and the final 
grade, without considering the differences 
between classes. The descriptive statis¬ 
tics on various measures are displayed in 


Table 5. Descriptive Statistics on 
Different Measures. 



Average 

Std. Dev. 

Grade 

92.1080 

7.62763 

Syllabus (times accessed) 

11.5000 

8.15370 

Posting (number posted) 

48.5000 

37.79892 

Skipped_weeks 

3.2386 

2.92038 

Delayed days 

5.2092 

3.54226 


Table 5, and the results of the correlation 
analysis are presented in Table 6. 

As Table 6 shows, the number of times 
the course syllabus was accessed is not 
significantly correlated with the final 
grade, although at the class level analysis, 
there was one class, i.e., the winter 2015 
Introductory class, in which correlation 
between the two was significant. That sig¬ 
nificance may be a special case for that 
particular class, in which it happened that 
those who checked the course syllabus 
more often got a higher grade finally. 

The number of days delayed to access 
the weekly materials has a negative cor¬ 
relation with the final grade, and the cor¬ 
relation is significant at p = 0.036 level. 
This result indicates that in general, more 
promptly accessing the weekly posted 
learning materials (less delaying) would 
lead to a higher final grade. However, the 
number of weeks that were skipped did not 
have a significant correlation with the final 
grade. 

The number of postings also has a 
strong, positive correlation with the final 
grade: the more postings, the higher the 
final grade. This correlation is significant 
only on borderline of 0.05 level, at p = 
0.056. 


Table 6. The Results of Correlation Analysis 
between Various Measures and Final Grade. 


Syllabus 

Posting 

Skipped_wks 

Delayed_days 

Grade Pearson Correlation 0.080 (0.229*) 

0.171 (0.056) 

0.086 (0.213) 

-0.193 (0.036) 


Notes: ‘Significance (p), one-tailed 
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Discussion 

Access to Course Syllabus—Does 
Students’ Access to the Course Syllabus 
Relate to Their Learning Performance? 

As the guideline for the course, the syl¬ 
labus is an important tool for students to 
get organized for the course, which in turn 
should be helpful for learning. Therefore, 
it was reasonable to assume that the fre¬ 
quency of accessing the course syllabus 
during the semester would be significantly 
associated with the final grade: the more 
accesses, the higher the grade. This was 
the case with only one class, however. At 
the general level, no significant correla¬ 
tions was found between access to sylla¬ 
bus and grade. 

The difference between the results from 
the class level analysis and that from the 
general level analysis, may be due to dif¬ 
ferent data size, and different class dynam¬ 
ics. In general the frequency of accessing 
syllabus on the course site did not seem to 
be associated with the final grade. How¬ 
ever, this does not exclude special cases/ 
classes that there could be significant cor¬ 
relation between this measure and learning 
performance, as it was in one of the classes 
in this study. 

Therefore, the answer to the first re¬ 
search question tends to be negative. In 
general, there is no significant association. 
However, there might be special cases or 
classes in which there would be a signifi¬ 
cant association: the more access the high¬ 
er the final grade. One possibility that there 
is no significant association is because the 
syllabus file is downloadable. Some stu¬ 
dents might have downloaded the file to 
their local computer, and when consult¬ 
ing the syllabus, they accessed the local 
file, without going online. The statistics, 
however, does not include local accesses. 
The result therefore suggests that using 
the number of access to a file that is down¬ 
loadable may not be an accurate measure 
to use. Since no previous research was 
done using this measure, future studies are 


needed to further investigate the effect of 
this interaction behavior. When conduct¬ 
ing such studies, access to the downloaded 
local syllabus file should be included. 

Access to Weekly Posted Learning 
Materials—Does the Access to Weekly 
Learning Materials Relate to Learning 
Performance? 

Two measures were used for this inter¬ 
action behavior: the number of skipped 
weeks and the number of days delayed to 
access for the weeks not skipped. The two 
scenarios are discussed separately below. 

The number of skipped weeks. Based on 
the experience from the traditional face- 
to-face learning settings, the assumption 
for this measure was that there would be a 
significantly negative association between 
the number of skipped weeks and the final 
grade: the more weeks when the posted 
learning materials would be skipped, the 
lower the final grade. This assumption did 
not hold true. The results show that the 
students’ final grade did not relate to the 
number of skipped weeks, from both the 
class level and the general level analyses. 
The data shows that the students’ behav¬ 
iors varied dramatically in each class. 
Many of them skipped many weekly class 
materials without an impact on their final 
good grade. 

There may be two explanations for 
this result. First, the student who skipped 
weekly learning materials might be fa¬ 
miliar with the content knowledge of that 
week, and therefore felt confident to skip 
that week’s posted learning materials. 

Second, not accessing the weekly post¬ 
ed learning materials does not mean the 
student did not learn: it might be the case 
that although the posted learning materials 
were skipped, the student still did required 
readings and used the textbook, without 
needing to go over the learning materials 
provided by the instructor. 

While this finding does not confirm the 
assumption that the more skipped weeks 
the lower the final grade, it does provide a 
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perspective to confirm the benefit of flex¬ 
ibility that is offered by online learning. 
Different students can tailor their learning 
based on their personal needs. 

The number of days delayed to access 
the learning materials. The results for this 
measure are mixed: while there was no 
significant correlation between the num¬ 
ber of days delayed and the final grade 
at the class level, the aggregated data did 
show a significant correlation between this 
measure and the final grade, at 0.05 level 
{p = 0.039). This result indicates that the 
more days are delayed, the lower the final 
grade would be. The reason that there was 
no significant correlation at the class level 
perhaps was due to the smaller data size. 

Significant or not, in both cases, similar 
to the skipped weeks measure, the correla¬ 
tions are negative, which reflects the na¬ 
ture of the relationship in the assumption 
that the more days delayed, the lower the 
final grade. 

The significantly negative correlation is 
understandable: not accessing in a timely 
fashion the weekly posted learning materi¬ 
als exhibits that the learning activities are 
not well organized, and thus not well dis¬ 
ciplined. Dror (2008) discussed the impor¬ 
tance of properly acquiring information 
for learning, from the cognition’s point 
of view, and using technology to enhance 
learners’ proper access to information 
(learning materials). 

In fact, an examination of the delays 
found a pattern among those who delayed 
access to weekly learning materials: de¬ 
lays most of time were “crammed access,” 
as described in Spivey and McMillan 
(2013): the student would wait for a few 
weeks to access the accumulated, more 
than one week’s learning materials at one 
time. From the student’s point of view, this 
may seem to save time and efforts: learn¬ 
ing would occur just once for more than 
one week, perhaps a few weeks, instead 
of spending time and efforts on learning 
every week. However, the result shows 
that this is not a healthy approach to learn¬ 
ing, and would lead to decreased learning 


performance. This result extends the find¬ 
ing from Spivey and McMillan (2013) to 
online students, that “cramming” not only 
“did not have positive impact,” but actu¬ 
ally having a significantly negative impact 
on learning performance. 

In summary, regarding research ques¬ 
tion 2, “Does student access to weekly 
learning materials posted by the instruc¬ 
tor relate to learning performance,” the 
answer is yes, if using the measure of the 
days delayed, and no, if using the measure 
of skipped weeks. 

The Number of Postings on Discussion 
Board—Does the Use of the Discussion 
Board on the Course Site Relate to 
Learning Performance? 

The feeling of isolation has long been 
recognized as a drawback for online learn¬ 
ing. Discussion boards are designed to 
help students communicate with the class, 
to overcome the feeling of isolation, and 
to help learning. Presumably, more fre¬ 
quent use of the discussion board, demon¬ 
strated through more postings, would lead 
to a better learning experience, and thus 
better performance (a higher final grade). 
Testing on 155 first-year female students, 
Alghamdi (2013) found that posting to a 
Blackboard discussion board was positive¬ 
ly related to improvement to the test scores 
among students. In this study, there were 
positive correlations between the number 
of postings and learning performance, but 
the correlation was not significant at the 
class level: none of the four classes had a 
significant correlation between the number 
of postings and the final grade. 

However, with the increased data size 
at the general level, the correlation was 
significant on borderline, at p = 0.056, and 
the correlation was positive. Therefore, 
the answer to research question 3 tends to 
be positive: The more postings, the higher 
the final grade. It is reasonable to expect 
a more significant correlation if an even 
larger data set would be available. This 
will need to be tested in future studies. 
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Implications of the Research 

While the significant correlation be¬ 
tween the access to syllabus and the final 
grade was just a special case in one class, 
the significant correlations between the 
delayed access to weekly learning ma¬ 
terials and the final grade, and between 
the number of postings on the discussion 
board and the final grade, have implica¬ 
tions from two perspectives: one is the stu¬ 
dents’ interaction behaviors on the course 
sites and the other is the technology sup¬ 
port provided by the learning management 
system. The two perspectives call for dif¬ 
ferent solutions to address the related is¬ 
sues in online education. 

From the perspective of student be¬ 
haviors, the findings provide empirical 
evidence to the “self-discipline” issue in e- 
leaming, and call for interventions to help 
online students. 

Delayed access to weekly learning ma¬ 
terials is similar to physically attending 
classes once every few weeks in the face- 
to-face environment. This is rare in face- 
to-face classrooms with a class size simi¬ 
lar to the ones investigated in this study, 
because such behavior is observable and 
will be under pressure from peer students 
and the instructor. The online learning en¬ 
vironment seems to make it easier for stu¬ 
dents to have this behavior. Consequently, 
this behavior could negatively affect a stu¬ 
dent’s learning performance. 

Measures have to be taken to help on¬ 
line students avoid such behavior. One 
way that is easier to implement in the on¬ 
line environment is to add weekly small 
quizzes into the curriculum, so that the 
students will have to go over each week’s 
learning materials in order to pass the 
week’s quiz to get credit. Quizzes can be 
set up on the learning management system 
to be graded automatically, so that there 
will be no extra burden on the instructor’s 
part. A considerable initial effort to set up 
the quiz is needed, though. 

The finding that the number of postings 
is positively correlated with learning per¬ 


formance indicates that active participa¬ 
tion is very important for an online student 
to get a higher grade. This finding is not 
surprising. Flowever, it is common in re¬ 
ality that despite the encouragement from 
the instructor, many students would still 
treat the participation in class discussions 
as “optional,” not realizing that it is one 
requirement for a higher grade. For the 
classes investigated in this study, students 
were required to participate in discussions, 
and a small portion of the total credit was 
used as reward. Still, some students did 
not care much to participate. 

One possible way to improve the situ¬ 
ation is to let the students be aware of the 
direct link between the participation and 
their final grade for the course. Flopefully, 
if a student wants a better grade, this will 
motivate the student to participate more in 
class discussions. Periodically providing 
feedback to students about their participa¬ 
tion, based on the numbers recorded by the 
learning management system, may also be 
helpful. 

From the technological perspective, 
although the study did not have the data 
regarding the ease of use of the technol¬ 
ogy (students investigated in the study 
were assumed to have the required high 
speed internet connection and necessary 
computing equipment, as the condition for 
being admitted into the online program), 
it is reasonable to suspect that the access 
and participation problems found in the 
study be partially attributed to technologi¬ 
cal issues, based on an examination of the 
Blackboard system. First, the system is 
designed mainly for desktop computers. A 
mobile version is available. Its functional¬ 
ity, however, is very limited: many parts 
of a course site are still the desktop ver¬ 
sion when accessed in the mobile version. 
For example, the assignment submission 
page is still the desktop version, making 
it difficult to read on a mobile device. This 
potentially hinders students’ access to the 
course material or tools when a student 
would be in mobile status, such as when 
traveling or being away from desktop 
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computers, which is not uncommon in an 
online environment. In fact, along with 
the popularity of smart mobile devices 
these years, many students are used to m- 
leaming, or using mobile devices to access 
learning resources (Veley, 2014). 

Second, the discussion board on the 
Blackboard system did not really support 
synchronous discussion sessions, lack¬ 
ing features that can be found in today’s 
social media or collaboration tools, such 
as instant chatting and voice messaging. 
Employing synchronous online discus¬ 
sion might help engage students. Voice 
messaging or chatting is particularly con¬ 
venient for mobile devices where typing 
is difficult. In the Blackboard system, the 
user can only post and respond to mes¬ 
sages asynchronously. Many of today’s 
students are social media users and used 
to using different tools to communicate in 
their own communities. It is not difficult to 
imagine that given the available learning 
management system functions, students 
might be reluctant to access course sites 
because of the backward system functions. 

Therefore, the findings of the study 
also indicate that there is large room for 
improving technology support for online 
education. Advanced technologies that 
people are used to should be incorporat¬ 
ed into learning management systems to 
better support students’ online learning 
needs. Mobile versions of learning man¬ 
agement systems should be enhanced to 
support m-leaming. 

Conclusion 

Online education is developing rapidly 
worldwide. This study investigated the re¬ 
lationships between students’ behaviors 
on online course sites and their learning 
performance, by analyzing the log data 
from four IT classes that were hosted on 
the Blackboard system. The study pro¬ 
vides the empirical evidence that in gen¬ 
eral, the number of days delayed to access 
the weekly posted learning materials has a 
significantly negative impact on learning 


performance, and the number of postings 
has a positive association on learning per¬ 
formance, significant on borderline. In a 
specific case, the frequency of accessing 
the course syllabus was significantly cor¬ 
related, positively, with learning perfor¬ 
mance. 

The findings confirmed the challenge 
in online education: how to engage online 
students in class participation? The impli¬ 
cations of the findings are discussed. On 
one hand, the data revealed the problemat¬ 
ic students’ behaviors on course sites, and 
suggestions are made to address the issue 
of delayed access to learning material and 
the issue of participation in class discus¬ 
sions: Weekly quizzes for the former and 
motivating the students with a potentially 
higher grade, plus providing periodic 
feedback to students, for the latter. These 
suggestions, however, are based purely 
on students’ behavior. On the other hand, 
from the technological point of view, the 
functions of the Blackboard system were 
examined, and the potential problems 
of the learning management system that 
might contribute to the identified behav¬ 
ior issues in the study are discussed. The 
current version of Blackboard system is 
lagging behind today’s social medial and 
communication software in terms of some 
advanced features. This lack of techno¬ 
logical support to students is suspected of 
hindering students’ participation in online 
classes. The study suggests enhancing mo¬ 
bile versions of learning management sys¬ 
tems to support m-learning, and develop¬ 
ing and adopting advanced functions such 
as instant chatting and voice messaging in 
learning management systems to support 
synchronous learning. 

It should be noted that the issues inves¬ 
tigated in the study are just a small aspect 
of online education. Online education is a 
complex process, involving many factors 
that could affect students’ learning perfor¬ 
mance. There are also multiple technolo¬ 
gies and modes that can be applied in on¬ 
line education. Different solutions may be 
helpful in addressing the problems identi- 
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fled. For example, the affordability of the 
needed technology may be a problem for 
some underprivileged students. Different 
delivery methods for course materials may 
be used, such as surface mail. But this is 
another model of education and it warrants 
an investigation of its own. For those who 
can afford the needed technology, more 
advanced technologies are needed to en¬ 
able students a more convenient access 
to learning resources. M-leaming should 
be convenient for those who use mobile 
devices frequently, and thus should be 
enhanced in online education. This study 
is limited to just the case using the Black¬ 
board learning system, and with its basic 
features. All other scenarios mentioned 
above deserve future studies. 

Future research also needs to consider 
other human factors that are important to 
learning, such as learner’s intelligence, 
prior knowledge, level of motivation, and 
learner’s cognitive and learning style, etc., 
as Poulova and Simonova (2012) pointed 
out. Attention should also be paid to un¬ 
derstanding of the nature of the technol¬ 
ogy use, so that students’ behaviors could 
be better understood (Bennett & Maton, 
2010). Only through continuous research 
and practice, can online education in gen¬ 
eral and students’ learning experience in 
particular, be improved. 
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